ebab7d0da53c51438720d60c6e2e467c - /page/wikibook/entry.mu

Micron Document

`:top

In `F33f`_`[statistics`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Statistics]`_`f, the `!backfitting algorithm`! is a simple iterative procedure used to fit a `F33f`_`[generalized additive model`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Generalized_additive_model]`_`f. It was introduced in 1985 by `F33f`_`[Leo Breiman`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Leo_Breiman]`_`f and `F33f`_`[Jerome Friedman`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Jerome_H._Friedman]`_`f along with generalized additive models. In most cases, the backfitting algorithm is equivalent to the `F33f`_`[Gauss–Seidel method`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Gauss–Seidel]`_`f, an algorithm used for solving a certain `F33f`_`[linear system of equations`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Linear_system_of_equations]`_`f.

>>Contents

• `F0af`_`[Algorithm`#algorithm]`_`f

• `F0af`_`[Motivation`#motivation]`_`f

• `F0af`_`[Explicit derivation for two dimensions`#explicit-derivation-for-two-dimensions]`_`f

• `F0af`_`[Issues`#issues]`_`f

• `F0af`_`[Modified algorithm`#modified-algorithm]`_`f

• `F0af`_`[References`#references]`_`f

• `F0af`_`[External links`#external-links]`_`f

-─

>>Algorithm

Additive models are a class of `F33f`_`[non-parametric regression`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Nonparametric_regression]`_`f models of the form:

Y i = α α + ∑ ∑ j = 1 p f j ( X i j ) + ϵ ϵ i {\\displaystyle Y_{i}=\\alpha +\\sum _{j=1}^{p}f_{j}(X_{ij})+\\epsilon _{i}}

where each X 1 , X 2 , … … , X p {\\displaystyle X_{1},X_{2},\\ldots ,X_{p}} is a variable in our p {\\displaystyle p} -dimensional predictor X {\\displaystyle X} , and Y {\\displaystyle Y} is our outcome variable. ϵ ϵ {\\displaystyle \\epsilon } represents our inherent error, which is assumed to have mean zero. The f j {\\displaystyle f_{j}} represent unspecified smooth functions of a single X j {\\displaystyle X_{j}} . Given the flexibility in the f j {\\displaystyle f_{j}} , we typically do not have a unique solution: α α {\\displaystyle \\alpha } is left unidentifiable as one can add any constants to any of the f j {\\displaystyle f_{j}} and subtract this value from α α {\\displaystyle \\alpha } . It is common to rectify this by constraining

∑ ∑ i = 1 N f j ( X i j ) = 0 {\\displaystyle \\sum _{i=1}^{N}f_{j}(X_{ij})=0} for all j {\\displaystyle j}

leaving

α α = 1 / N ∑ ∑ i = 1 N y i {\\displaystyle \\alpha =1/N\\sum _{i=1}^{N}y_{i}}

necessarily.

The backfitting algorithm is then:

`B100`F9d9 Initialize`f`b

`B100`F9d9`f`b

`B100`F9d9 α`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 =`f`b

`B100`F9d9 1`f`b

`B100`F9d9`f`b

`B100`F9d9 /`f`b

`B100`F9d9`f`b

`B100`F9d9 N`f`b

`B100`F9d9`f`b

`B100`F9d9 ∑`f`b

`B100`F9d9`f`b

`B100`F9d9 i`f`b

`B100`F9d9 =`f`b

`B100`F9d9 1`f`b

`B100`F9d9`f`b

`B100`F9d9 N`f`b

`B100`F9d9`f`b

`B100`F9d9 y`f`b

`B100`F9d9`f`b

`B100`F9d9 i`f`b

`B100`F9d9`f`b

`B100`F9d9 ,`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 j`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 ≡`f`b

`B100`F9d9 0`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle {\\hat {\\alpha }}=1/N\\sum _{i=1}^{N}y_{i},{\\hat {f_{j}}}\\equiv 0}`f`b

`B100`F9d9`f`b

`B100`F9d9,`f`b

`B100`F9d9`f`b

`B100`F9d9 ∀`f`b

`B100`F9d9 j`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle \\forall j}`f`b

`B100`F9d9`f`b

`B100`F9d9 Do until`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 j`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle {\\hat {f_{j}}}}`f`b

`B100`F9d9`f`b

`B100`F9d9 converge:`f`b

`B100`F9d9 For each predictor j:`f`b

`B100`F9d9 (a)`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 j`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 ←`f`b

`B100`F9d9`f`b

`B100`F9d9 Smooth`f`b

`B100`F9d9`f`b

`B100`F9d9 [`f`b

`B100`F9d9 {`f`b

`B100`F9d9`f`b

`B100`F9d9 y`f`b

`B100`F9d9`f`b

`B100`F9d9 i`f`b

`B100`F9d9`f`b

`B100`F9d9 −`f`b

`B100`F9d9`f`b

`B100`F9d9 α`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 −`f`b

`B100`F9d9`f`b

`B100`F9d9 ∑`f`b

`B100`F9d9`f`b

`B100`F9d9 k`f`b

`B100`F9d9 ≠`f`b

`B100`F9d9 j`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 k`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 (`f`b

`B100`F9d9`f`b

`B100`F9d9 x`f`b

`B100`F9d9`f`b

`B100`F9d9 i`f`b

`B100`F9d9 k`f`b

`B100`F9d9`f`b

`B100`F9d9 )`f`b

`B100`F9d9`f`b

`B100`F9d9 }`f`b

`B100`F9d9`f`b

`B100`F9d9 1`f`b

`B100`F9d9`f`b

`B100`F9d9 N`f`b

`B100`F9d9`f`b

`B100`F9d9 ]`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle {\\hat {f_{j}}}\\leftarrow {\\text{Smooth}}[\\lbrace y_{i}-{\\hat {\\alpha }}-\\sum _{k\\neq j}{\\hat {f_{k}}}(x_{ik})\\rbrace _{1}^{N}]}`f`b

`B100`F9d9`f`b

`B100`F9d9 (backfitting step)`f`b

`B100`F9d9 (b)`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 j`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 ←`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 j`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 −`f`b

`B100`F9d9 1`f`b

`B100`F9d9`f`b

`B100`F9d9 /`f`b

`B100`F9d9`f`b

`B100`F9d9 N`f`b

`B100`F9d9`f`b

`B100`F9d9 ∑`f`b

`B100`F9d9`f`b

`B100`F9d9 i`f`b

`B100`F9d9 =`f`b

`B100`F9d9 1`f`b

`B100`F9d9`f`b

`B100`F9d9 N`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 j`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 (`f`b

`B100`F9d9`f`b

`B100`F9d9 x`f`b

`B100`F9d9`f`b

`B100`F9d9 i`f`b

`B100`F9d9 j`f`b

`B100`F9d9`f`b

`B100`F9d9 )`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle {\\hat {f_{j}}}\\leftarrow {\\hat {f_{j}}}-1/N\\sum _{i=1}^{N}{\\hat {f_{j}}}(x_{ij})}`f`b

`B100`F9d9`f`b

`B100`F9d9 (mean centering of estimated function)`f`b

where Smooth {\\displaystyle {\\text{Smooth}}} is our smoothing operator. This is typically chosen to be a `F33f`_`[cubic spline smoother`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Smoothing_spline]`_`f but can be any other appropriate fitting operation, such as:

• local `F33f`_`[polynomial regression`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Polynomial_regression]`_`f

• `F33f`_`[kernel smoothing`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Kernel_smoothing]`_`f methods

• more complex operators, such as surface smoothers for second and higher-order interactions

In theory, step `!(b)`! in the algorithm is not needed as the function estimates are constrained to sum to zero. However, due to numerical issues this might become a problem in practice.`:cite-ref-1[`F5bf`_`[1`#cite-note-1]`_`f]

>>Motivation

If we consider the problem of minimizing the expected squared error:

min α α , f j E [ ( Y − − α α − − ∑ ∑ j = 1 p f j ( X j ) ) 2 ] {\\displaystyle \\min _{\\alpha ,f_{j}}\\ \\mathbb {E} [(Y-\\alpha -\\sum _{j=1}^{p}f_{j}(X_{j}))^{2}]}

There exists a unique solution by the theory of projections given by:

f i ( X i ) = E [ Y − − α α − − ∑ ∑ j ≠ ≠ i p f j ( X j ) ∣ ∣ X i ] {\\displaystyle f_{i}(X_{i})=\\mathbb {E} [Y-\\alpha -\\sum _{j\\neq i}^{p}f_{j}(X_{j})\\mid X_{i}]}

for `*i`* = 1, 2, ..., `*p`*.

This gives the matrix interpretation:

( I P 1 ⋯ ⋯ P 1 P 2 I ⋯ ⋯ P 2 ⋮ ⋮ ⋱ ⋱ ⋮ ⋮ P p ⋯ ⋯ P p I ) ( f 1 ( X 1 ) f 2 ( X 2 ) ⋮ ⋮ f p ( X p ) ) = ( P 1 Y P 2 Y ⋮ ⋮ P p Y ) {\\displaystyle {\\begin{pmatrix}I&P_{1}&\\cdots &P_{1}\\\\P_{2}&I&\\cdots &P_{2}\\\\\\vdots &&\\ddots &\\vdots \\\\P_{p}&\\cdots &P_{p}&I\\end{pmatrix}}{\\begin{pmatrix}f_{1}(X_{1})\\\\f_{2}(X_{2})\\\\\\vdots \\\\f_{p}(X_{p})\\end{pmatrix}}={\\begin{pmatrix}P_{1}Y\\\\P_{2}Y\\\\\\vdots \\\\P_{p}Y\\end{pmatrix}}}

where P i ( ⋅ ⋅ ) = E ( ⋅ ⋅ | X i ) {\\displaystyle P_{i}(\\cdot )=\\mathbb {E} (\\cdot |X_{i})} . In this context we can imagine a smoother matrix, S i {\\displaystyle S_{i}} , which approximates our P i {\\displaystyle P_{i}} and gives an estimate, S i Y {\\displaystyle S_{i}Y} , of E ( Y | X ) {\\displaystyle \\mathbb {E} (Y|X)}

( I S 1 ⋯ ⋯ S 1 S 2 I ⋯ ⋯ S 2 ⋮ ⋮ ⋱ ⋱ ⋮ ⋮ S p ⋯ ⋯ S p I ) ( f 1 f 2 ⋮ ⋮ f p ) = ( S 1 Y S 2 Y ⋮ ⋮ S p Y ) {\\displaystyle {\\begin{pmatrix}I&S_{1}&\\cdots &S_{1}\\\\S_{2}&I&\\cdots &S_{2}\\\\\\vdots &&\\ddots &\\vdots \\\\S_{p}&\\cdots &S_{p}&I\\end{pmatrix}}{\\begin{pmatrix}f_{1}\\\\f_{2}\\\\\\vdots \\\\f_{p}\\end{pmatrix}}={\\begin{pmatrix}S_{1}Y\\\\S_{2}Y\\\\\\vdots \\\\S_{p}Y\\end{pmatrix}}}

or in abbreviated form

S ^ ^ f = Q Y {\\displaystyle {\\hat {S}}f=QY\\,}

An exact solution of this is infeasible to calculate for large `*np`*, so the iterative technique of backfitting is used. We take initial guesses f j ( 0 ) {\\displaystyle f_{j}^{(0)}} and update each f j ( ℓ ℓ ) {\\displaystyle f_{j}^{(\\ell )}} in turn to be the smoothed fit for the residuals of all the others:

f j ^ ^ ( ℓ ℓ ) ← ← Smooth [ { y i − − α α ^ ^ − − ∑ ∑ k ≠ ≠ j f k ^ ^ ( x i k ) } 1 N ] {\\displaystyle {\\hat {f_{j}}}^{(\\ell )}\\leftarrow {\\text{Smooth}}[\\lbrace y_{i}-{\\hat {\\alpha }}-\\sum _{k\\neq j}{\\hat {f_{k}}}(x_{ik})\\rbrace _{1}^{N}]}

Looking at the abbreviated form it is easy to see the backfitting algorithm as equivalent to the `F33f`_`[Gauss–Seidel method`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Gauss–Seidel]`_`f for linear smoothing operators `*S`*.

>>Explicit derivation for two dimensions

Following,`:cite-ref-2[`F5bf`_`[2`#cite-note-2]`_`f] we can formulate the backfitting algorithm explicitly for the two dimensional case. We have:

f 1 = S 1 ( Y − − f 2 ) , f 2 = S 2 ( Y − − f 1 ) {\\displaystyle f_{1}=S_{1}(Y-f_{2}),f_{2}=S_{2}(Y-f_{1})}

If we denote f ^ ^ 1 ( i ) {\\displaystyle {\\hat {f}}_{1}^{(i)}} as the estimate of f 1 {\\displaystyle f_{1}} in the `*i`*th updating step, the backfitting steps are

f ^ ^ 1 ( i ) = S 1 [ Y − − f ^ ^ 2 ( i − − 1 ) ] , f ^ ^ 2 ( i ) = S 2 [ Y − − f ^ ^ 1 ( i ) ] {\\displaystyle {\\hat {f}}_{1}^{(i)}=S_{1}[Y-{\\hat {f}}_{2}^{(i-1)}],{\\hat {f}}_{2}^{(i)}=S_{2}[Y-{\\hat {f}}_{1}^{(i)}]}

By induction we get

f ^ ^ 1 ( i ) = Y − − ∑ ∑ α α = 0 i − − 1 ( S 1 S 2 ) α α ( I − − S 1 ) Y − − ( S 1 S 2 ) i − − 1 S 1 f ^ ^ 2 ( 0 ) {\\displaystyle {\\hat {f}}_{1}^{(i)}=Y-\\sum _{\\alpha =0}^{i-1}(S_{1}S_{2})^{\\alpha }(I-S_{1})Y-(S_{1}S_{2})^{i-1}S_{1}{\\hat {f}}_{2}^{(0)}}

and

f ^ ^ 2 ( i ) = S 2 ∑ ∑ α α = 0 i − − 1 ( S 1 S 2 ) α α ( I − − S 1 ) Y + S 2 ( S 1 S 2 ) i − − 1 S 1 f ^ ^ 2 ( 0 ) {\\displaystyle {\\hat {f}}_{2}^{(i)}=S_{2}\\sum _{\\alpha =0}^{i-1}(S_{1}S_{2})^{\\alpha }(I-S_{1})Y+S_{2}(S_{1}S_{2})^{i-1}S_{1}{\\hat {f}}_{2}^{(0)}}

If we set f ^ ^ 2 ( 0 ) = 0 {\\displaystyle {\\hat {f}}_{2}^{(0)}=0} then we get

f ^ ^ 1 ( i ) = Y − − S 2 − − 1 f ^ ^ 2 ( i ) = [ I − − ∑ ∑ α α = 0 i − − 1 ( S 1 S 2 ) α α ( I − − S 1 ) ] Y {\\displaystyle {\\hat {f}}_{1}^{(i)}=Y-S_{2}^{-1}{\\hat {f}}_{2}^{(i)}=[I-\\sum _{\\alpha =0}^{i-1}(S_{1}S_{2})^{\\alpha }(I-S_{1})]Y}

f ^ ^ 2 ( i ) = [ S 2 ∑ ∑ α α = 0 i − − 1 ( S 1 S 2 ) α α ( I − − S 1 ) ] Y {\\displaystyle {\\hat {f}}_{2}^{(i)}=[S_{2}\\sum _{\\alpha =0}^{i-1}(S_{1}S_{2})^{\\alpha }(I-S_{1})]Y}

Where we have solved for f ^ ^ 1 ( i ) {\\displaystyle {\\hat {f}}_{1}^{(i)}} by directly plugging out from f 2 = S 2 ( Y − − f 1 ) {\\displaystyle f_{2}=S_{2}(Y-f_{1})} .

We have convergence if ‖ ‖ S 1 S 2 ‖ ‖ < 1 {\\displaystyle \\|S_{1}S_{2}\\|<1} . In this case, letting f ^ ^ 1 ( i ) , f ^ ^ 2 ( i ) → f ^ ^ 1 ( ∞ ∞ ) , f ^ ^ 2 ( ∞ ∞ ) {\\displaystyle {\\hat {f}}_{1}^{(i)},{\\hat {f}}_{2}^{(i)}{\\xrightarrow {}}{\\hat {f}}_{1}^{(\\infty )},{\\hat {f}}_{2}^{(\\infty )}} :

f ^ ^ 1 ( ∞ ∞ ) = Y − − S 2 − − 1 f ^ ^ 2 ( ∞ ∞ ) = Y − − ( I − − S 1 S 2 ) − − 1 ( I − − S 1 ) Y {\\displaystyle {\\hat {f}}_{1}^{(\\infty )}=Y-S_{2}^{-1}{\\hat {f}}_{2}^{(\\infty )}=Y-(I-S_{1}S_{2})^{-1}(I-S_{1})Y}

f ^ ^ 2 ( ∞ ∞ ) = S 2 ( I − − S 1 S 2 ) − − 1 ( I − − S 1 ) Y {\\displaystyle {\\hat {f}}_{2}^{(\\infty )}=S_{2}(I-S_{1}S_{2})^{-1}(I-S_{1})Y}

We can check this is a solution to the problem, i.e. that f ^ ^ 1 ( i ) {\\displaystyle {\\hat {f}}_{1}^{(i)}} and f ^ ^ 2 ( i ) {\\displaystyle {\\hat {f}}_{2}^{(i)}} converge to f 1 {\\displaystyle f_{1}} and f 2 {\\displaystyle f_{2}} correspondingly, by plugging these expressions into the original equations.

>>Issues

The choice of when to stop the algorithm is arbitrary and it is hard to know a priori how long reaching a specific convergence threshold will take. Also, the final model depends on the order in which the predictor variables X i {\\displaystyle X_{i}} are fit.

As well, the solution found by the backfitting procedure is non-unique. If b {\\displaystyle b} is a vector such that S ^ ^ b = 0 {\\displaystyle {\\hat {S}}b=0} from above, then if f ^ ^ {\\displaystyle {\\hat {f}}} is a solution then so is f ^ ^ + α α b {\\displaystyle {\\hat {f}}+\\alpha b} is also a solution for any α α ∈ ∈ R {\\displaystyle \\alpha \\in \\mathbb {R} } . A modification of the backfitting algorithm involving projections onto the eigenspace of `*S`* can remedy this problem.

>>Modified algorithm

We can modify the backfitting algorithm to make it easier to provide a unique solution. Let V 1 ( S i ) {\\displaystyle {\\mathcal {V}}_{1}(S_{i})} be the space spanned by all the eigenvectors of `*S`*i that correspond to eigenvalue 1. Then any `*b`* satisfying S ^ ^ b = 0 {\\displaystyle {\\hat {S}}b=0} has b i ∈ ∈ V 1 ( S i ) ∀ ∀ i = 1 , … … , p {\\displaystyle b_{i}\\in {\\mathcal {V}}_{1}(S_{i})\\forall i=1,\\dots ,p} and ∑ ∑ i = 1 p b i = 0. {\\displaystyle \\sum _{i=1}^{p}b_{i}=0.} Now if we take A {\\displaystyle A} to be a matrix that projects orthogonally onto V 1 ( S 1 ) + ⋯ ⋯ + V 1 ( S p ) {\\displaystyle {\\mathcal {V}}_{1}(S_{1})+\\dots +{\\mathcal {V}}_{1}(S_{p})} , we get the following modified backfitting algorithm:

`B100`F9d9 Initialize`f`b

`B100`F9d9`f`b

`B100`F9d9 α`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 =`f`b

`B100`F9d9 1`f`b

`B100`F9d9`f`b

`B100`F9d9 /`f`b

`B100`F9d9`f`b

`B100`F9d9 N`f`b

`B100`F9d9`f`b

`B100`F9d9 ∑`f`b

`B100`F9d9`f`b

`B100`F9d9 1`f`b

`B100`F9d9`f`b

`B100`F9d9 N`f`b

`B100`F9d9`f`b

`B100`F9d9 y`f`b

`B100`F9d9`f`b

`B100`F9d9 i`f`b

`B100`F9d9`f`b

`B100`F9d9 ,`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 j`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 ≡`f`b

`B100`F9d9 0`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle {\\hat {\\alpha }}=1/N\\sum _{1}^{N}y_{i},{\\hat {f_{j}}}\\equiv 0}`f`b

`B100`F9d9`f`b

`B100`F9d9,`f`b

`B100`F9d9`f`b

`B100`F9d9 ∀`f`b

`B100`F9d9 i`f`b

`B100`F9d9 ,`f`b

`B100`F9d9 j`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle \\forall i,j}`f`b

`B100`F9d9`f`b

`B100`F9d9,`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 +`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 =`f`b

`B100`F9d9 α`f`b

`B100`F9d9 +`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 1`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 +`f`b

`B100`F9d9 ⋯`f`b

`B100`F9d9 +`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 p`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle {\\hat {f_{+}}}=\\alpha +{\\hat {f_{1}}}+\\dots +{\\hat {f_{p}}}}`f`b

`B100`F9d9`f`b

`B100`F9d9 Do until`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 j`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle {\\hat {f_{j}}}}`f`b

`B100`F9d9`f`b

`B100`F9d9 converge:`f`b

`B100`F9d9 Regress`f`b

`B100`F9d9`f`b

`B100`F9d9 y`f`b

`B100`F9d9 −`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 +`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle y-{\\hat {f_{+}}}}`f`b

`B100`F9d9`f`b

`B100`F9d9 onto the space`f`b

`B100`F9d9`f`b

`B100`F9d9 V`f`b

`B100`F9d9`f`b

`B100`F9d9 1`f`b

`B100`F9d9`f`b

`B100`F9d9 (`f`b

`B100`F9d9`f`b

`B100`F9d9 S`f`b

`B100`F9d9`f`b

`B100`F9d9 i`f`b

`B100`F9d9`f`b

`B100`F9d9 )`f`b

`B100`F9d9 +`f`b

`B100`F9d9 ⋯`f`b

`B100`F9d9 +`f`b

`B100`F9d9`f`b

`B100`F9d9 V`f`b

`B100`F9d9`f`b

`B100`F9d9 1`f`b

`B100`F9d9`f`b

`B100`F9d9 (`f`b

`B100`F9d9`f`b

`B100`F9d9 S`f`b

`B100`F9d9`f`b

`B100`F9d9 p`f`b

`B100`F9d9`f`b

`B100`F9d9 )`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle {\\mathcal {V}}_{1}(S_{i})+\\dots +{\\mathcal {V}}_{1}(S_{p})}`f`b

`B100`F9d9`f`b

`B100`F9d9, setting`f`b

`B100`F9d9`f`b

`B100`F9d9 a`f`b

`B100`F9d9 =`f`b

`B100`F9d9 A`f`b

`B100`F9d9 (`f`b

`B100`F9d9 Y`f`b

`B100`F9d9 −`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 +`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 )`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle a=A(Y-{\\hat {f_{+}}})}`f`b

`B100`F9d9`f`b

`B100`F9d9 For each predictor j:`f`b

`B100`F9d9 Apply backfitting update to`f`b

`B100`F9d9`f`b

`B100`F9d9 (`f`b

`B100`F9d9 Y`f`b

`B100`F9d9 −`f`b

`B100`F9d9 a`f`b

`B100`F9d9 )`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle (Y-a)}`f`b

`B100`F9d9`f`b

`B100`F9d9 using the smoothing operator`f`b

`B100`F9d9`f`b

`B100`F9d9 (`f`b

`B100`F9d9 I`f`b

`B100`F9d9 −`f`b

`B100`F9d9`f`b

`B100`F9d9 A`f`b

`B100`F9d9`f`b

`B100`F9d9 i`f`b

`B100`F9d9`f`b

`B100`F9d9 )`f`b

`B100`F9d9`f`b

`B100`F9d9 S`f`b

`B100`F9d9`f`b

`B100`F9d9 i`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle (I-A_{i})S_{i}}`f`b

`B100`F9d9`f`b

`B100`F9d9, yielding new estimates for`f`b

`B100`F9d9`f`b

`B100`F9d9 f`f`b

`B100`F9d9`f`b

`B100`F9d9 j`f`b

`B100`F9d9`f`b

`B100`F9d9 ^`f`b

`B100`F9d9`f`b

`B100`F9d9 {\\displaystyle {\\hat {f_{j}}}}`f`b

`B100`F9d9`f`b

>>References

`:cite-note-1`!1.`! `F0af`_`[↑`#cite-ref-1]`_`f `F33f`_`[Hastie, Trevor`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Trevor_Hastie]`_`f, `F33f`_`[Robert Tibshirani`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Robert_Tibshirani]`_`f and Jerome Friedman (2001). `*The Elements of Statistical Learning: Data Mining, Inference, and Prediction`*. Springer, `F33f`_`[ISBN`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=ISBN_(identifier)]`_`f 0-387-95284-5.

`:cite-note-2`!2.`! `F0af`_`[↑`#cite-ref-2]`_`f Härdle, Wolfgang; et al. (June 9, 2004). "Backfitting". Archived from the original on 2015-05-10. Retrieved 2015-08-19.

• `:citerefbreiman-l-friedman-j-h-1985`aBreiman, L. & Friedman, J. H. (1985). "Estimating optimal transformations for multiple regression and correlations (with discussion)". `*Journal of the American Statistical Association`*. `!80`! (391): 580–619. `F33f`_`[doi`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=Doi_(identifier)]`_`f:10.2307/2288473. `F33f`_`[JSTOR`:/page/wikibook/entry.mu`zim=wikipedia_en_all_nopic_2025-08.zim|entry_path=JSTOR_(identifier)]`_`f 2288473.

• `:citerefhastie-t-j-tibshirani-r-j-1990`aHastie, T. J. & Tibshirani, R. J. (1990). "Generalized Additive Models". `*Monographs on Statistics and Applied Probability`*. `!43`!.

• `:citerefh-rdle-wolfgang2004`aHärdle, Wolfgang; et al. (June 9, 2004). "Backfitting". Archived from the original on 2015-05-10. Retrieved 2015-08-19.

>>External links

• R Package for GAM backfitting

• R Package for BRUTO backfitting

`c`F0af`_`[↑ Back to top`#top]`_`f`a